In what scenarios does it break (is it model dependent) or is it purely backend dependent? Also, are there any drawbacks to it quality/perf wise?

If it's an all round improvement, rather than overcomplicate with too many flags, would it be better to just force-enable it for supported backends? or is it conditionally broken?

It directly depends on the implementation of a specific operation, so it's unlikely to crash on a specific model (except in the sense that a model family which doesn't use that operation would of course always work, but I think all currently do).

In terms of performance, it's a bit too early to be sure. When it helps, it helps a lot, but it's not clear if it could get worse for some backend+model+hardware combinations. Quality seems to be the same, although outputs may vary a bit; in my tests, the VAE output changes less than it does with tiling.

Another possibility could be enabling the flags unconditionally for known-to-be-working cases, but including an environment variable to work around possible issues (either forcing it on to test a new backend, or off to avoid a special case).

it's not clear if it could get worse for some backend+model+hardware combinations

Already found an example - my own card! VAE timings for rendering a 512x512 SDXL image, radv versus amdvlk:

It's even more dramatic for diffusion-conv-direct : ~20% worse on radv, 5x worse on amdvlk.

Alright how about this:

• we can add the extra new toggle for SD flash attention
• we add a new toggle for conv direct, but we combine both of them into a single flag for simplicity. The flag toggles them both. How about --sdconvdirect (avoid 'dir' as that usually means directory here)

we add a new toggle for conv direct, but we combine both of them into a single flag for simplicity. The flag toggles them both. How about --sdconvdirect (avoid 'dir' as that usually means directory here)

A single boolean option would be a bit annoying, to be honest :-) My card gets 20% slower for the diffusion step, so it could be slower overall for high step counts, even if the VAE is much faster.

But a non-boolean sdconvdirect would be fine. I've implemented one with three values: disabled, enabled, and 'vaeonly' to enable it only for the VAE (I don't think we'll find a case where only the the VAE would be slower, but it's easy to add an extra value for that if you wish).

Anyway let me know when its ready to review

Should be good to go now... except maybe for the combobox control. I tried to follow the other labeled controls as examples; it works, but you'll probably want to adjust its position and padding. It's being called only at one point right now, but I may make use of it in another (unrelated) PR.

You added sd_diffusion_convdir_var but it is not used anywhere

Cleaned up that var, and rebased on top of concedo_experimental.

Edit: while working on #1692 , I started to think on and off could be better than enabled and disabled; or maybe off, vaeonly and full. I don't really have a strong preference, but maybe it'd be best to choose values that could be the same for other options, for consistency?

I think the disambiguation is good. So long as the commands are clear it's good. Generally I try to avoid enabled/on/off/disabled flags because that is implied with store true, i.e. --usemmap means mmap is enabled. So if 3 options are needed I think keeping it explicit is ok